卷积神经网络（CNN）通常是使用基于随机梯度下降（SGD）优化技术训练的。现有的SGD优化器通常会遭受最小值和最低振荡的过度损失。在本文中，我们提出了一种新方法，以下内容称为Adainject，以将二阶时刻注入一阶时刻，以称为梯度下降优化器。具体而言，参数的短期更改被用作重量，以在更新规则中注入二阶时刻。 Adainject优化器控制参数更新，避免了最小值的过度换档，并减少了最小值接近的振荡。提出的方法本质上是通用的，可以与任何现有的SGD优化器集成。通过直观地解释了Anainject优化器的有效性以及一些玩具示例。我们还显示了拟议的基于注射的优化器的收敛性。此外，我们通过广泛的实验与最新的优化器（即Adaminject，diffgradinject，radaminject和Adabeliefinject在四个基准数据集中）一起描述了ADAIN方法的功效。实验中使用了不同的CNN模型。在CIFAR10数据集上使用resnext29模型，使用diffgradinject Optimizer观察到TOP-1分类错误率$ 16.54 \％$的最高提高。总体而言，我们通过提出的ADAIN方法观察到现有优化器的性能提高非常有希望。该代码可在：\ url {https://github.com/shivram1987/adainject}中获得。

Three main points: 1. Data Science (DS) will be increasingly important to heliophysics; 2. Methods of heliophysics science discovery will continually evolve, requiring the use of learning technologies [e.g., machine learning (ML)] that are applied rigorously and that are capable of supporting discovery; and 3. To grow with the pace of data, technology, and workforce changes, heliophysics requires a new approach to the representation of knowledge.

We describe a Physics-Informed Neural Network (PINN) that simulates the flow induced by the astronomical tide in a synthetic port channel, with dimensions based on the Santos - S\~ao Vicente - Bertioga Estuarine System. PINN models aim to combine the knowledge of physical systems and data-driven machine learning models. This is done by training a neural network to minimize the residuals of the governing equations in sample points. In this work, our flow is governed by the Navier-Stokes equations with some approximations. There are two main novelties in this paper. First, we design our model to assume that the flow is periodic in time, which is not feasible in conventional simulation methods. Second, we evaluate the benefit of resampling the function evaluation points during training, which has a near zero computational cost and has been verified to improve the final model, especially for small batch sizes. Finally, we discuss some limitations of the approximations used in the Navier-Stokes equations regarding the modeling of turbulence and how it interacts with PINNs.

Model counting is a fundamental problem which has been influential in many applications, from artificial intelligence to formal verification. Due to the intrinsic hardness of model counting, approximate techniques have been developed to solve real-world instances of model counting. This paper designs a new anytime approach called PartialKC for approximate model counting. The idea is a form of partial knowledge compilation to provide an unbiased estimate of the model count which can converge to the exact count. Our empirical analysis demonstrates that PartialKC achieves significant scalability and accuracy over prior state-of-the-art approximate counters, including satss and STS. Interestingly, the empirical results show that PartialKC reaches convergence for many instances and therefore provides exact model counting performance comparable to state-of-the-art exact counters.

The findable, accessible, interoperable, and reusable (FAIR) data principles have provided a framework for examining, evaluating, and improving how we share data with the aim of facilitating scientific discovery. Efforts have been made to generalize these principles to research software and other digital products. Artificial intelligence (AI) models -- algorithms that have been trained on data rather than explicitly programmed -- are an important target for this because of the ever-increasing pace with which AI is transforming scientific and engineering domains. In this paper, we propose a practical definition of FAIR principles for AI models and create a FAIR AI project template that promotes adherence to these principles. We demonstrate how to implement these principles using a concrete example from experimental high energy physics: a graph neural network for identifying Higgs bosons decaying to bottom quarks. We study the robustness of these FAIR AI models and their portability across hardware architectures and software frameworks, and report new insights on the interpretability of AI predictions by studying the interplay between FAIR datasets and AI models. Enabled by publishing FAIR AI models, these studies pave the way toward reliable and automated AI-driven scientific discovery.

We introduce Patch Aligned Contrastive Learning (PACL), a modified compatibility function for CLIP's contrastive loss, intending to train an alignment between the patch tokens of the vision encoder and the CLS token of the text encoder. With such an alignment, a model can identify regions of an image corresponding to a given text input, and therefore transfer seamlessly to the task of open vocabulary semantic segmentation without requiring any segmentation annotations during training. Using pre-trained CLIP encoders with PACL, we are able to set the state-of-the-art on the task of open vocabulary zero-shot segmentation on 4 different segmentation benchmarks: Pascal VOC, Pascal Context, COCO Stuff and ADE20K. Furthermore, we show that PACL is also applicable to image-level predictions and when used with a CLIP backbone, provides a general improvement in zero-shot classification accuracy compared to CLIP, across a suite of 12 image classification datasets.

In this paper the implementation of piezoelectrics to a state-of-the-art wafer gripper is investigated. The objective is to propose and validate a solution method, which includes a mechanical design and control system, to achieve at least 5% damping for two eigenmodes of a wafer gripper. This objective serves as a 'proof of concept' to show the possibilities of implementing a state-of-the-art damping method to an industrial application, which in turn can be used to dampen different thin structures. The coupling relation between the piezoelectrics and their host structure were used to design the placement of the piezoelectric patches, together with modal analysis data of the a state-of-the-art wafer gripper. This data had been measured through an experimental setup. Active damping has been succesfully implemented onto the wafer gripper where positive position feedback (PPF) is used as a control algorithm to dampen two eigenmodes.

Skull Stripping is a requisite preliminary step in most diagnostic neuroimaging applications. Manual Skull Stripping methods define the gold standard for the domain but are time-consuming and challenging to integrate into pro-cessing pipelines with a high number of data samples. Automated methods are an active area of research for head MRI segmentation, especially deep learning methods such as U-Net architecture implementations. This study compares Vanilla, Residual, and Dense 2D U-Net architectures for Skull Stripping. The Dense 2D U-Net architecture outperforms the Vanilla and Residual counterparts by achieving an accuracy of 99.75% on a test dataset. It is observed that dense interconnections in a U-Net encourage feature reuse across layers of the architecture and allow for shallower models with the strengths of a deeper network.

Classifiers and generators have long been separated. We break down this separation and showcase that conventional neural network classifiers can generate high-quality images of a large number of categories, being comparable to the state-of-the-art generative models (e.g., DDPMs and GANs). We achieve this by computing the partial derivative of the classification loss function with respect to the input to optimize the input to produce an image. Since it is widely known that directly optimizing the inputs is similar to targeted adversarial attacks incapable of generating human-meaningful images, we propose a mask-based stochastic reconstruction module to make the gradients semantic-aware to synthesize plausible images. We further propose a progressive-resolution technique to guarantee fidelity, which produces photorealistic images. Furthermore, we introduce a distance metric loss and a non-trivial distribution loss to ensure classification neural networks can synthesize diverse and high-fidelity images. Using traditional neural network classifiers, we can generate good-quality images of 256$\times$256 resolution on ImageNet. Intriguingly, our method is also applicable to text-to-image generation by regarding image-text foundation models as generalized classifiers. Proving that classifiers have learned the data distribution and are ready for image generation has far-reaching implications, for classifiers are much easier to train than generative models like DDPMs and GANs. We don't even need to train classification models because tons of public ones are available for download. Also, this holds great potential for the interpretability and robustness of classifiers. Project page is at \url{https://classifier-as-generator.github.io/}.

Out-of-distribution detection is crucial to the safe deployment of machine learning systems. Currently, the state-of-the-art in unsupervised out-of-distribution detection is dominated by generative-based approaches that make use of estimates of the likelihood or other measurements from a generative model. Reconstruction-based methods offer an alternative approach, in which a measure of reconstruction error is used to determine if a sample is out-of-distribution. However, reconstruction-based approaches are less favoured, as they require careful tuning of the model's information bottleneck - such as the size of the latent dimension - to produce good results. In this work, we exploit the view of denoising diffusion probabilistic models (DDPM) as denoising autoencoders where the bottleneck is controlled externally, by means of the amount of noise applied. We propose to use DDPMs to reconstruct an input that has been noised to a range of noise levels, and use the resulting multi-dimensional reconstruction error to classify out-of-distribution inputs. Our approach outperforms not only reconstruction-based methods, but also state-of-the-art generative-based approaches.